With recent increases in network bandwidth, the ubiquitous interconnectedness of users through the global Internet, and the increasing volume of digital data processed by business and consumer users, the demands for network-based transfer of bulk data (files and directories) are ever growing. In particular, users desire to transfer larger files, over networks of ever higher bandwidths, and at ever longer distances.
Such data transfer paths not only experience high bottleneck bandwidths and round-trip delays due to geographical distance, but they also experience periods of packet losses, and variable delays due to the media itself (e.g. wireless), and to variable and sometimes excessive, traffic congestion.
Conventional bulk data transfer protocols based on the Transmission Control Protocol (TCP) suffer from severe performance limitations over typical global Internet paths, due to the poor performance of TCP over networks with high bandwidth—delay products. Much attention has focused on implementations and alternative transport protocols for improving performance (transfer speed and bandwidth utilization) for bulk data transfer on high-bandwidth, high-delay networks. However, current approaches offer improved throughputs and bandwidth utilization primarily on links in the Internet core, which have relatively low bit error rates (BER) and have an abundance of bandwidth, avoiding traffic congestion. However, the majority of user data transfers span the network edge-to-edge, and not only experience high round-trip delays due to geographical distance, but also experience periods of packet losses and variable delay characteristic of the typical “edge” network. On typical edge networks, current approaches fail to achieve full bandwidth utilization, suffer from variable throughputs as congestion increases, and can not provide sufficient guarantees on transfer times required by time-critical business processes and demanding consumer users. Furthermore, in the limited cases where current approaches do improve throughput, they do so at the expense of fair sharing of bandwidth with other network applications, and provide the end user no control over the bandwidth sharing. The end user is forced to choose between a poorly performing but “fair” standard TCP implementation, or an alternative new protocol that provides improved throughput in limited cases but at the expense of bandwidth fairness. While this may be acceptable in the Internet core, it is not acceptable on the often over-subscribed edge networks where data transfers are admitted to networks with limited available bandwidth. There is a need in the art for a system for data transfer that addresses the foregoing concerns and provides improved throughput, predictable transfer speeds independent of the network distance or congestion (and associated delays and packet losses), automatic full utilization of bandwidth, and the ability to share bandwidth proportionally with other traffic when no bandwidth is unused, taking into account both current and future implementations of the TCP protocol.